There is increasing interest in General Purpose Graphics Processing Units (GPGPUs) that include a plurality of streaming multiprocessors. GPGPUs are GPUs that may also be used for other types of processing, such as image processing and scientific processing. Background information on GPGPUs and streaming multiprocessors are described in the book, GPU Gems 2: Programming Techniques for High-Performance Graphics and General-Purpose Computation, editors Matt Pharr and Randima Fernando, Pearson Education (2005), the contents of which are hereby incorporated by reference.
Advances in semiconductor technology permit a GPGPU to have a large number of computation units on a single die. As described in chapter 29 of GPU Gems 2, in a streaming programming model, all data is represented as a stream, where a stream is an ordered set of data of the same data type. Kernels operate on entire streams of elements. In a stream programming model, applications are constructed by chaining multiple kernels together. Since kernels operate on entire streams, stream elements can be processed in parallel using parallel multiprocessors. One model for a high performance GPU includes a task parallel organization, in that all kernels can be run simultaneously, and a data level parallelism in that data is processed in parallel computation units.
In a GPGPU having a large number of computation units the individual computation units will issue memory requests to a shared memory, such as a graphics frame buffer. However, memory controllers typically permit only a limited number of memory access sizes. Additionally, memory controllers typically provide efficient memory accesses when accessing a limited range of dynamic random access memory (DRAM) addresses at one time, such as addresses on a common DRAM page. Consequently, in some situations a GPGPU may not be capable of efficiently servicing the different memory requests of the different computation units.
In light of the above-described problems the apparatus, system, and method of the present invention was developed.